NATURAL LANGUAGE UNDERSTANDING TASKS

Document-Level Neural Machine Translation Using BERT as Context Encoder (2020.aacl-srw)

Challenge:	Recent research points to knowledge distillation as a potential solution for NLU tasks.
Approach:	They propose a training approach that distills large finetuned LMs into a small network using unlabeled training examples.
Outcome:	The proposed approach outperforms BERT training approaches while using 300 times fewer parameters.

Copied to clipboard

Zhiyu Guo, Minh Le Nguyen

Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text (D19-60)

Challenge:	Large-scale pre-trained representations such as BERT have been widely used in many natural language understanding tasks.
Approach:	They propose to use BERT as a context encoder to achieve document-level contextual information, which is then integrated into both the encoder and decoder.
Outcome:	The proposed model outperforms strong document-level machine translation baselines on BLEU score and captures document- level context information to boost translation performance.

Copied to clipboard

Ian Porada, Kaheer Suleman, Jackie Chi Kit Cheung

An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks (2020.aacl-main)

Challenge:	Existing work on modeling semantic plausibility has focused on physical plausability but distributional methods fail when tested in supervised settings.
Approach:	They propose to use large pretrained language models to model plausibility in supervised settings by extracting attested events from a large corpus and injecting explicit commonsense knowledge into a distributional model.
Outcome:	The proposed model is effective in modeling plausibility in a supervised setting.

Copied to clipboard

Kyubyong Park, Joohong Lee, Seongbo Jang, Dawoon Jung

Syntactic Text Processing

Challenge:	Traditionally, tokenization is the very first step in most text processing works.
Approach:	They propose to use morphological segmentation followed by BPE for Korean NLP tasks . they empirically examine what is the best tokenization strategy for Korean to/from English .
Outcome:	The proposed approach is best for Korean to/from English machine translation and natural language understanding tasks.

Triple-Hybrid Energy-based Model Makes Better Calibrated Natural Language Understanding Models (2023.eacl-main)

Copied to clipboard

Haotian Xu, Yingying Zhang

Challenge:	In-distribution (ID) miscalibration and out-of-difference (OOD) detection are main concerns for pre-trained language models.
Approach:	They propose a triple-hybrid EBM which combines the benefits of classifier, conditional generative model and marginal generative models altogether.
Outcome:	The proposed model outperforms previous methods in terms of ID calibration and OOD detection by a large margin while maintaining competitive accuracy.

Noise Robust Named Entity Understanding for Voice Assistants (2021.naacl-industry)

Copied to clipboard

Deepak Muralidharan, Joel Ruben Antony Moniz, Sida Gao, Xiao Yang, Justine Kao, Stephen Pulman, Atish Kothari, Ray Shen, Yinying Pan, Vivek Kaul, Mubarak Seyed Ibrahim, Gang Xiang, Nan Dun, Yidan Zhou, Andy O, Yuan Zhang, Pooja Chitkara, Xuan Wang, Alkesh Patel, Kushal Tayal, Roger Zheng, Peter Grasch, Jason D Williams, Lin Li

HybridBERT - Making BERT Pretraining More Efficient Through Hybrid Mixture of Attention Mechanisms (2024.naacl-srw)

Challenge:	Named Entity Recognition and Entity Linking are challenging for voice assistants . utterances are relatively short, so there is not much context to help disambiguate .
Approach:	They propose a Named Entity Understanding system that combines NER and EL in a joint reranking module.
Outcome:	The proposed framework improves NER accuracy by up to 3.13% and EL accuracy by 3.6% in F1 score . it also leads to better accuracies in other natural language understanding tasks .

Copied to clipboard

Gokul Srinivasagan, Simon Ostermann

Task-specific Compression for Multi-task Language Models using Attribution-based Pruning (2023.findings-eacl)

Challenge:	Pretrained transformer-based language models have produced state-of-the-art performance in most natural language understanding tasks.
Approach:	They propose two hybrid architectures that combine self-attention and additive attention mechanisms with sub-layer normalization to achieve double the pretraining accuracy of a vanilla-BERT baseline.
Outcome:	The proposed architectures outperform BERT-base on two downstream tasks while accelerating inference.

Copied to clipboard

Nakyeong Yang, Yunah Jang, Hwanhee Lee, Seohyeong Jeong, Kyomin Jung

Active Learning by Acquiring Contrastive Examples (2021.emnlp-main)

Challenge:	Existing compression methods for multi-task language models use large number of parameter parameters even when performing only a specific task.
Approach:	They propose a training-free compression method for multi-task language models using pruning method . they use an attribution method to determine which neurons are essential for performing a specific task .
Outcome:	The proposed method outperforms baseline pruning methods on six widely-used datasets.

Copied to clipboard

Katerina Margatina, Giorgos Vernikos, Loïc Barrault, Nikolaos Aletras

Paraphrasing in Affirmative Terms Improves Negation Understanding (2024.acl-short)

Challenge:	Using uncertainty and diversity sampling, active learning acquisition functions select difficult and diverse data points from a pool of unlabeled data.
Approach:	They propose an active learning acquisition function that selects contrastive examples from unlabeled data.
Outcome:	The proposed approach performs better or equal to the best performing baseline on all tasks, on both in-domain and out-of-domain data.

Copied to clipboard

MohammadHossein Rezaei, Eduardo Blanco

Text Generation

Challenge:	Negation is a common linguistic phenomenon in human languages . however, language models face challenges with negation in many tasks .
Approach:	They propose to incorporate affirmative interpretations into models to make them more robust against negation.
Outcome:	The proposed models are more robust against negation when negation is present in input . the proposed models can be used to analyze large corpus and natural language understanding tasks .

Parameter-Efficient Tuning with Special Token Adaptation (2023.eacl-main)

Copied to clipboard

Xiaocong Yang, James Y. Huang, Wenxuan Zhou, Muhao Chen

Efficient Learning of Multiple NLP Tasks via Collective Weight Factorization on BERT (2022.findings-naacl)

Challenge:	a recent study shows that parameter-efficient tuning is a challenge for multitask deployments.
Approach:	They propose a parameter-efficient tuning technique that only updates a small subset of parameters when adapting a pretrained model to downstream tasks.
Outcome:	The proposed method achieves comparable performance to fine-tuning in natural language understanding tasks including text classification and NER with only 0.029% of parameters trained.

Copied to clipboard

Christos Papadopoulos, Yannis Panagakis, Manolis Koubarakis, Mihalis Nicolaou

Think Wider, Detect Sharper: Reinforced Reference Coverage for Document-Level Self-Contradiction Detection (2025.emnlp-main)

Challenge:	Existing methods to fine-tune a model for multiple tasks require a large amount of memory and computing power.
Approach:	They propose to factorize the weighs of a pre-trained Transformer model to improve training efficiency across multiple tasks by using BERT-Large as an instantiation of the Transformer and the GLUE as the evaluation benchmark.
Outcome:	The proposed method matches or improves the original fine-tuned model’s performance for each task while effectively decreasing parameter requirements by two orders of magnitude.

Copied to clipboard

Yuhao Chen, Yuanjie Lyu, Shuochen Liu, Chao Zhang, Junhui Lv, Tong Xu

Effective Use of Transformer Networks for Entity Tracking (D19-1)

Challenge:	Recent approaches to document-level contradiction detection (DSCD) only gain marginal improvement and often introduce inconsistencies across repeated responses.
Approach:	They propose a method that combines supervised fine-tuning and reinforcement learning to enhance document-level contradiction detection (DSCD) they propose to use a task-specific reward function to expand the model’s reasoning scope, boosting both accuracy and consistency.
Outcome:	The proposed method significantly boosts Llama 3.1-8B-Instruct’s accuracy from 38.5% to 51.1%, and consistency from 59.6% to76.2%.

Copied to clipboard

Aditya Gupta, Greg Durrett

Scientia Potentia Est—On the Role of Knowledge in Computational Argumentation (2022.tacl-1)

Challenge:	Existing pre-trained language models for entity-related tasks are not able to handle the nuances of procedural text.
Approach:	They propose to use pre-trained transformer networks to track entities in procedural text by restructuring input to focus on a particular entity.
Outcome:	The proposed models outperform baseline models on ingredient detection in recipes and QA over scientific processes on two different tasks.

Copied to clipboard

Anne Lauscher, Henning Wachsmuth, Iryna Gurevych, Goran Glavaš

Few-shot initializing of Active Learner via Meta-Learning (2022.findings-emnlp)

Challenge:	Existing research on argumentation models does not provide a systematic overview of the types of knowledge required in CA tasks.
Approach:	They propose a taxonomy of the types of knowledge required in CA tasks . authors propose exploitation of these knowledge types for four main research areas .
Outcome:	The proposed taxonomy proposes a systematic overview of the types of knowledge required in CA tasks.

Copied to clipboard

Zi Long Zhu, Vikrant Yadav, Zubair Afzal, George Tsatsaronis

Low-Resource NLP

Challenge:	Recent advances in few-shot and zero-shot learning have limited performance in domain specific applications.
Approach:	They propose to initialize an active learner with meta-learned parameters and generate task dependent softmax weights for active learning.
Outcome:	The proposed method performs better than the baseline at low budget, the authors show . they show that adding meta-learned learning rates and generating the softmax have negative consequences .

Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)

Copied to clipboard

Yu Cao, Wei Bi, Meng Fang, Dacheng Tao

Dialogue Systems & Conversational Agents

Challenge:	Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks.
Approach:	They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines.
Outcome:	The proposed models deliver higher relevance with dialogue history than baselines.

An Analysis of Negation in Natural Language Understanding Corpora (2022.acl-short)

Copied to clipboard

Md Mosharaf Hossain, Dhivya Chinnappa, Eduardo Blanco

Challenge:	Using annotator-generated examples, one can evaluate systems with synthetic language that is not representative of language in the wild.
Approach:	They analyze negation in eight popular corpora spanning six natural language understanding tasks.
Outcome:	The proposed corpora have few negations compared to general-purpose English and are often unimportant . state-of-the-art transformers obtain significantly worse results with instances that contain negation, especially if the negations are important.

Comparing Test Sets with Item Response Theory (2021.acl-long)

Copied to clipboard

Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, Samuel R. Bowman

Linguistic Theories

Challenge:	Recent results from large pretrained models show that many datasets are saturated and unlikely to detect further progress.
Approach:	They evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples.
Outcome:	The proposed datasets are saturated and unlikely to detect future improvements.

Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining (P18-2)

Copied to clipboard

Eyal Shnarch, Carlos Alzate, Lena Dankin, Martin Gleize, Yufang Hou, Leshem Choshen, Ranit Aharonov, Noam Slonim

SERENGETI: Massively Multilingual Language Models for Africa (2023.findings-acl)

Challenge:	Obtaining high quality labeled data for natural language understanding tasks is slow, error-prone, complicated and expensive.
Approach:	They propose a method to blend weak and strong labeled data during the training of neural networks using a topic-dependent evidence detection dataset.
Outcome:	The proposed method improves the training of neural networks when a small amount of labeled data is available.

Copied to clipboard

Ife Adebara, AbdelRahim Elmadany, Muhammad Abdul-Mageed, Alcides Alcoba Inciarte

What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks? (2021.acl-long)

Challenge:	Pretrained models acquire valuable, generalizable linguistic information during pretraining and have advanced the state of the art on task-specific finetuning.
Approach:	They develop a set of massively multilingual language models that covers 517 African languages and language varieties.
Outcome:	The proposed models outperform 4 models that cover 4-23 African languages on eight natural language understanding tasks, achieving 82.27 average F_1.

Copied to clipboard

Nikita Nangia, Saku Sugawara, Harsh Trivedi, Alex Warstadt, Clara Vania, Samuel R. Bowman

Question Answering

Challenge:	Despite the importance of datasets for natural language understanding, there has been little attention on crowdsourcing methods for collecting datasets.
Approach:	They compare the effectiveness of crowdsourcing methods for boosting NLU example difficulty with training crowdworkers instead of expert judgments.
Outcome:	The proposed method is ineffective for boosting NLU example difficulty, but it is not effective for training crowdworkers and qualifying workers based on expert judgments.

Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning (2020.coling-main)

Copied to clipboard

Daniel Grießhaber, Johannes Maucher, Ngoc Thang Vu

TextHide: Tackling Data Privacy in Language Understanding Tasks (2020.findings-emnlp)

Challenge:	Recent work has explored the suitability of pre-trained language models in low resource settings with less than 1,000 training data points.
Approach:	They propose to use pool-based active learning to speed up training while keeping the cost of labeling new data constant.
Outcome:	The proposed model can be fine-tuned to optimize for low-resource settings while keeping the cost of labeling constant.

Copied to clipboard

Yangsibo Huang, Zhao Song, Danqi Chen, Kai Li, Sanjeev Arora

Hence, Socrates is mortal: A Benchmark for Natural Language Syllogistic Reasoning (2023.findings-acl)

Challenge:	Unsolved privacy challenges in distributed or federated learning are a challenge for many domains including Natural Language Processing.
Approach:	They propose a federated learning framework that adds an encryption step to prevent an eavesdropping attacker from recovering private text data.
Outcome:	The proposed model can effectively defend against attacks on shared gradients or representations and the averaged accuracy reduction is only 1.9%.

Copied to clipboard

Yongkang Wu, Meng Han, Yutao Zhu, Lei Li, Xinyu Zhang, Ruofei Lai, Xiaoguang Li, Yuanhang Ren, Zhicheng Dou, Zhao Cao

How Certain is Your Transformer? (2021.eacl-main)

Challenge:	SylloBase is a benchmark for syllogistic reasoning, a critical capability widely required in natural language understanding tasks, such as text entailment and question answering.
Approach:	They propose to use a benchmark to learn syllogistic reasoning on a set of templates and to use them to generate and understand slogisms.
Outcome:	The proposed benchmark covers a complete taxonomy of syllogism reasoning patterns, and contains both automatically and manually constructed samples.

Copied to clipboard

Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanin, Alexander Panchenko, Maxim Panov

MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning (2024.acl-long)

Challenge:	Obtaining reliable uncertainty estimations for such neural networks (NNs) is challenging due to the huge number of parameters in these deep learning models.
Approach:	They propose to use Monte Carlo dropout to estimate uncertainty for Transformer-based models and construct inexpensive estimates using Determinantal Point Processes.
Outcome:	The proposed estimates improve the quality of detection of error-prone instances.

Copied to clipboard

Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten Rijke, Zhumin Chen, Jiahuan Pei

Things not Written in Text: Exploring Spatial Commonsense from Visual Signals (2022.acl-long)

Challenge:	Large language models (LLMs) are the default paradigm for natural language processing (NLP) as the models’ scale and the diversity of tasks increase, fine-tuning becomes infeasible.
Approach:	They propose to freeze original pretrained weights and train a group of mini LoRAs with only a small number of parameters and reduce their rank by 8 times .
Outcome:	The proposed model uses fewer trainable parameters while maintaining a higher rank, thereby offering improved performance potential.

Copied to clipboard

Xiao Liu, Da Yin, Yansong Feng, Dongyan Zhao

Multimodality

Challenge:	Pretrained language models fail in many NLP tasks, but are ineffective in spatial commonsense reasoning.
Approach:	They propose a spatial commonsense benchmark that focuses on relative scales of objects and the positional relationship between people and objects under different actions.
Outcome:	The proposed framework outperforms pretrained models in answering spatial questions.

Document Modeling with External Attention for Sentence Extraction (P18-1)

Copied to clipboard

Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang

Information Extraction & Text Mining

Challenge:	Document modeling is essential to a variety of natural language understanding tasks.
Approach:	They propose to use external information to improve document modeling for sentence extraction problems.
Outcome:	The proposed model outperforms baseline models on document summarization and answer selection tasks and achieves state-of-the-art results on WikiQA and NewsQA.

Take a Closer Look at Multilinguality! Improve Multilingual Pre-Training Using Monolingual Corpora Only (2023.findings-emnlp)

Copied to clipboard

Jinliang Lu, Yu Lu, Jiajun Zhang

Leveraging Grammar Induction for Language Understanding and Generation (2024.findings-emnlp)

Challenge:	Recent studies have demonstrated remarkable cross-lingual capability of pre-trained language models . however, semantic alignments may be the reason behind such capability but remain under-explored.
Approach:	They propose token-level and semantic-level code-switched masked language modeling to improve cross-lingual interactions over mono-mPLMs without parallel sentences.
Outcome:	The proposed method outperforms mono-mPLMs on natural language understanding and unsupervised machine translation tasks.

Copied to clipboard

Jushi Kai, Shengyuan Hou, Yusheng Huang, Zhouhan Lin

Domain Adversarial Fine-Tuning as an Effective Regularizer (2020.findings-emnlp)

Challenge:	Existing grammar induction methods do not provide sufficient performance in downstream tasks.
Approach:	They propose an unsupervised grammar induction method for language understanding and generation using a grammar parser and a syntactic mask.
Outcome:	The proposed method performs better on from-scratch and pre-trained scenarios.

Copied to clipboard

Giorgos Vernikos, Katerina Margatina, Alexandra Chronopoulou, Ion Androutsopoulos

Data Annealing for Informal Language Understanding Tasks (2020.findings-emnlp)

Challenge:	Existing fine-tuning techniques can degrade general-domain representations . however, fine-timing can lead to catastrophic forgetting of knowledge .
Approach:	They propose a new regularization technique that complements the task-specific loss used during fine-tuning with an adversarial objective.
Outcome:	Empirical results show that AFTER improves performance on various natural language understanding tasks compared to standard fine-tuning.

Copied to clipboard

Jing Gu, Zhou Yu

LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression (2020.coling-main)

Challenge:	Existing models that improve formal and informal language understanding tasks do not transfer to informal data directly.
Approach:	They propose a data annealing transfer learning procedure to bridge the performance gap on informal natural language understanding tasks.
Outcome:	The proposed procedure outperforms state-of-the-art models on three common tasks.

Copied to clipboard

Yihuan Mao, Yujing Wang, Chufan Wu, Chen Zhang, Yang Wang, Quanlu Zhang, Yaming Yang, Yunhai Tong, Jing Bai

You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments (2024.naacl-long)

Challenge:	Existing models that use knowledge distillation are memory-intensive and latency-prohibitive . Existing solutions that use this knowledge distilling framework are expensive .
Approach:	They propose a solution that uses weight pruning, matrix factorization and knowledge distillation to learn a smaller model.
Outcome:	The proposed model reduces the training overheads by an order of magnitude on public datasets while preserving state-of-the-art accuracy.

Copied to clipboard

Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, David Jurgens

KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters (2024.findings-acl)

Challenge:	Large Language Models (LLMs) are popular for research in social sciences . currently, prompting LLMs is insufficient to accurately and reliably capture model perceptions, and we discuss potential alternatives to improve this.
Approach:	They construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes and a set of questions containing minor variations.
Outcome:	The proposed model can generate answers and negate statements in a consistent and robust manner.

Copied to clipboard

SungHo Kim, Juhyeong Park, Yeachan Kim, SangKeun Lee

Designing Templates for Eliciting Commonsense Knowledge from Pretrained Sequence-to-Sequence Models (2020.coling-main)

Challenge:	Existing pre-trained language models for Korean have overlooked character representation principles . a new framework for Korean PLMs uses the invention principles of Hangeul to represent letters .
Approach:	They propose a framework that uses Hangeul's invention principles to represent character in Korean . their method outperforms the state-of-the-art Korean PLM by 2.11% in five tasks .
Outcome:	The proposed method outperforms the state-of-the-art Korean PLM by 2.11% in five Korean natural language understanding tasks.

Copied to clipboard

Jheng-Hong Yang, Sheng-Chieh Lin, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, Jimmy Lin

Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes (2026.findings-eacl)

Challenge:	Existing approaches to extract implicit knowledge from pretrained models are still unclear.
Approach:	They propose to use a template-based approach to extract implicit knowledge for commonsense reasoning on multiple-choice questions.
Outcome:	The proposed template can be extended to other MC tasks with contexts such as supporting facts in open-book question answering settings.

Copied to clipboard

Abdullah Al Monsur, Nitesh Vamshi Bommisetty, Gene Louis Kim

Distill and Replay for Continual Language Learning (2020.coling-main)

Challenge:	Decoder-only models dominate the event detection literature, but their unidirectional attention mechanism has been a roadblock in getting strong performance on embedding.
Approach:	They propose to use Macro-F1 as a more representative measure of a model’s ability across the long-tail of event types to improve their models' performance.
Outcome:	The proposed model improves on the decoder-only models, showing that low-rank Adaptation can be an effective tool to enhance LLMs’ performance on long-tailed event classes.

Copied to clipboard

Jingyuan Sun, Shaonan Wang, Jiajun Zhang, Chengqing Zong

Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankers (2024.naacl-long)

Challenge:	Existing models fail to isolate acquired knowledge and forget previously learned tasks when learning in a stream where data distribution may shift.
Approach:	They propose a framework that distills knowledge and replays experience from previous tasks when fitting on a new task.
Outcome:	The proposed framework outperforms state-of-the-art models in continuously learning tasks of the same type but from different domains, as well as tasks of different types.

Copied to clipboard

Yuan Wang, Xuyang Wu, Hsin-Tai Wu, Zhiqiang Tao, Yi Fang

Imagination-Augmented Natural Language Understanding (2022.naacl-main)

Challenge:	Recent studies have shown that Large Language Models (LLMs) are more efficient in natural language understanding tasks.
Approach:	They evaluate large language models (LLMs) using a TREC Fair Ranking dataset . they assess fairness from both user and content perspectives .
Outcome:	The proposed model outperforms the existing models in the fair ranking task.

Copied to clipboard

Yujie Lu, Wanrong Zhu, Xin Wang, Miguel Eckstein, William Yang Wang

Multimodality

Challenge:	Existing methods for Natural Language Understanding focus on textual signals, which hinders models from learning efficiently from limited data samples.
Approach:	They propose an Imagination-Augmented Cross-modal Encoder to solve natural language understanding tasks from a novel learning perspective.
Outcome:	The proposed learning paradigm bridges the gap between human and agent language understanding in both linguistic and perceptual procedures.

Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection (2024.naacl-long)

Copied to clipboard

Jianfeng He, Hang Su, Jason Cai, Igor Shalyminov, Hwanjun Song, Saab Mansour

Information Extraction & Text Mining

Challenge:	Semi-supervised dialogue summarization (SSDS) leverages model-generated summaries to reduce reliance on human-labeled data.
Approach:	They propose a scoring approach that encapsulates three primary dimensions of summarization model quality.
Outcome:	The proposed method reduces reliance on human-labeled data and improves the performance of summarization models.

Hypernymy Detection for Low-Resource Languages via Meta Learning (2020.acl-main)

Copied to clipboard

Changlong Yu, Jialong Han, Haisong Zhang, Wilfred Ng

ARCH: Efficient Adversarial Regularized Training with Caching (2021.findings-emnlp)

Challenge:	Existing studies focus on monolingual hypernymy detection on high-resource languages, but few investigate low-resourced scenarios.
Approach:	They propose to combine high-resource languages to solve low-resourced hypernymy detection problem . they extensively compare three joint training paradigms and propose meta learning .
Outcome:	The proposed method significantly improves performance of extremely low-resource languages by preventing over-fitting on small datasets.

Copied to clipboard

Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Task Calibration: Calibrating Large Language Models on Inference Tasks (2025.findings-acl)

Challenge:	Existing approaches to regularize models require generating a perturbation for each sample in each epoch.
Approach:	They propose an adversarial regularization method where perturbations are generated and cached once every several epochs.
Outcome:	The proposed method significantly eases the computational burden (saves up to 70% of computational time) it produces a notably better (in most of the tasks) or comparable model generalization.

Copied to clipboard

Yingjie Li, Yun Luo, Xiaotian Xie, Yue Zhang

Zero-Shot Cross-Lingual Transfer with Meta Learning (2020.emnlp-main)

Challenge:	Large language models (LLMs) have shown impressive zero-shot performance on inference tasks, however, they may suffer from spurious correlations between input texts and output labels, which limits their ability to reason based purely on general language understanding.
Approach:	They propose a zero-shot and inference-only calibration method inspired by mutual information which recovers LLM performance through task reformulation.
Outcome:	The proposed calibration method improves on 13 benchmarks and prompt templates and can be integrated with other calibration methods.

Copied to clipboard

Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, Isabelle Augenstein

Low-Resource NLP

Challenge:	There are more than 7,000 languages spoken in the world, over 90 of which have more than 10 million native speakers each.
Approach:	They propose to use meta-learning to train a model on multiple languages at the same time . they use standard supervised, zero-shot cross-lingual, and few-shot crosses-lingual settings for different natural language understanding tasks.
Outcome:	The proposed setup improves on the state-of-the-art for a total of 15 languages.

Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity (2024.lrec-main)

Copied to clipboard

Sho Hoshino, Akihiko Kato, Soichiro Murakami, Peinan Zhang

KALA: Knowledge-Augmented Language Model Adaptation (2022.naacl-main)

Challenge:	Using labeled NLI datasets for learning sentence embeddings leads to improved performance for natural language understanding tasks.
Approach:	They compare two data augmentation techniques for learning better sentence embeddings . they use a cross-lingual transfer technique that exploits English resources as training data to yield non-English sentence embeds as zero-shot inference .
Outcome:	The proposed techniques yield better performance on Japanese and Korean sentences.

Copied to clipboard

Minki Kang, Jinheon Baek, Sung Ju Hwang

Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding (2022.emnlp-main)

Challenge:	Pre-trained language models (PLMs) have proved to be effective on various natural language understanding tasks.
Approach:	They propose a domain adaption framework which modulates the intermediate hidden representations of PLMs with domain knowledge, consisting of entities and their relational facts.
Outcome:	The proposed framework outperforms adaptive pre-training on question answering and named entity recognition tasks on multiple datasets across different domains.

Copied to clipboard

Md Mosharaf Hossain, Eduardo Blanco

Explainability & Interpretability in NLP

Challenge:	Negation poses a challenge in many natural language understanding tasks . leveraging sentences with negation and affirmative interpretations is beneficial for many tasks involving humans .
Approach:	They propose to collect negated sentences and their affirmative interpretations and leverage them to build a plug-and-play neural generator that generates an affirmative interpreter.
Outcome:	The proposed method does not require manual effort and does not impact other tasks.

An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference (2024.findings-emnlp)

Copied to clipboard

Atsuki Yamaguchi, Aline Villavicencio, Nikolaos Aletras

Making Language Models Robust Against Negation (2025.naacl-long)

Challenge:	Cross-lingual vocabulary adaptation (CVA) methods have been proposed for adapting models to a target language . but effectiveness of these methods on increasing inference efficiency of generative large language models has not been explored.
Approach:	They propose to use cross-lingual vocabulary adaptation methods to adapt models to a target language to improve downstream performance.
Outcome:	The proposed methods significantly speed up models in four languages and four natural language understanding tasks.

Copied to clipboard

MohammadHossein Rezaei, Eduardo Blanco

NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? (2022.findings-emnlp)

Challenge:	Negation is a semantic phenomenon that alters an expression to convey the opposite meaning.
Approach:	They propose a self-supervised method to make language models more robust against negation by pre-training models.
Outcome:	The proposed task outperforms the off-the-shelf versions on nine negation-related benchmarks.

Copied to clipboard

Saadia Gabriel, Hamid Palangi, Yejin Choi

Text Classification

Challenge:	Existing work has explored adversarial example generation for natural language understanding tasks, but these examples are often unrealistic and diverge from the real-world data distributions.
Approach:	They propose a framework for adversarial example generation that is effective at fooling a given classifier and a generative model based on the key tokens from the first stage.
Outcome:	The proposed framework generalizes across domains and offers insights for future research on improving robustness of neural text classification models.

Sharpness-Aware Minimization with Dynamic Reweighting (2022.findings-emnlp)

Copied to clipboard

Wenxuan Zhou, Fangyu Liu, Huan Zhang, Muhao Chen

Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher (2022.coling-1)

Challenge:	Deep neural networks are often overparameterized and can overfit training data.
Approach:	They propose an adversarial weight minimization algorithm that conducts adversarials and finds a common adversaria per-batch.
Outcome:	The proposed algorithm finds a common adversarial weight perturbation per-batch.

Copied to clipboard

Mehdi Rezagholizadeh, Aref Jafari, Puneeth S.M. Saladi, Pranav Sharma, Ali Saheb Pasand, Ali Ghodsi

Leitner-Guided Memory Replay for Cross-lingual Continual Learning (2024.naacl-long)

Challenge:	Knowledge distillation (KD) is a powerful tool for deep learning applications.
Approach:	They propose a method which defines a smoother training path for the student by following the training footprints of the teacher rather than solely relying on distilling from a single mature fully-trained teacher.
Outcome:	The proposed technique is quite effective in mitigating the capacity-gap problem and the checkpoint search problem.

Copied to clipboard

Meryem M’hamdi, Jonathan May

Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding (2022.coling-1)

Challenge:	Various continual learning approaches have proposed to mitigate catastrophic forgetting by restricting the data buffer or limiting the data size of a model.
Approach:	They propose to use a human-inspired spaced-repetition technique to prioritize examples for cross-lingual continual learning.
Outcome:	The proposed approach significantly and consistently decreases forgetting while maintaining accuracy across natural language understanding tasks, language orders, and languages.

Copied to clipboard

Zhaoye Fei, Yu Tian, Yongkang Wu, Xinyu Zhang, Yutao Zhu, Zheng Liu, Jiawen Wu, Dejiang Kong, Ruofei Lai, Zhao Cao, Zhicheng Dou, Xipeng Qiu

SLM-Mod: Small Language Models Surpass LLMs at Content Moderation (2025.naacl-long)

Challenge:	Existing methods to learn downstream tasks by stitches skill block lack rationality and interpretation.
Approach:	They propose a hierarchical framework with a coarse-to-fine paradigm for generalized text representations from the large-scale corpus.
Outcome:	The proposed model learns basic language properties from all tasks and boosts performance on relevant tasks.

Copied to clipboard

Xianyang Zhan, Agam Goyal, Yilun Chen, Eshwar Chandrasekharan, Koustuv Saha

Mitigating Shortcut Learning with InterpoLated Learning (2025.acl-long)

Challenge:	Large language models (LLMs) are expensive to query in real-time and do not allow for a community-specific approach to content moderation.
Approach:	They propose to use small language models for community-specific content moderation tasks by fine-tuning and evaluating their performance against larger open- and closed-sourced models.
Outcome:	The proposed models outperform zero-shot LLMs in content moderation tasks with 11.5% higher accuracy and 25.7% higher recall across all communities.

Copied to clipboard

Michalis Korakakis, Andreas Vlachos, Adrian Weller

ALBETO and DistilBETO: Lightweight Spanish Language Models (2022.lrec-1)

Challenge:	Existing shortcut mitigation approaches are model-specific, difficult to tune, computationally expensive, and fail to improve learned representations.
Approach:	They propose to interpolate representations of majority examples to include features from intra-class minority examples with shortcut-mitigating patterns.
Outcome:	The proposed method improves minority generalization over ERM and state-of-the-art mitigation methods on multiple natural language understanding tasks while preserving accuracy on majority examples.

Copied to clipboard

José Cañete, Sebastian Donoso, Felipe Bravo-Marquez, Andrés Carvallo, Vladimir Araujo

An empirical analysis of existing systems and datasets toward general simple question answering (2020.coling-main)

Challenge:	Recent advances in pre-trained language models have made them more popular . however, there are still limited versions of these models for other languages .
Approach:	They present ALBETO and DistilBETO which are versions of ALBERT and DistillBERT pre-trained exclusively on Spanish corpora.
Outcome:	The proposed models outperform BETO and ALBERT on Spanish datasets . the models outpace BETO on MLDoc, PAWS-X, XNLI, MLQA, SQAC and XQuAD datasets.

Copied to clipboard

Namgi Han, Goran Topic, Hiroshi Noji, Hiroya Takamura, Yusuke Miyao

Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work? (2020.acl-main)

Challenge:	evaluators of simple factoid question answering using different datasets are not able to solve SimpleQuestions.
Approach:	They evaluate the progress of the field toward solving simple factoid questions over a knowledge base.
Outcome:	The proposed model is nearly solved on the most popular dataset, but not on the robustness of existing systems.

Copied to clipboard

Yada Pruksachatkun, Jason Phang, Haokun Liu, Phu Mon Htut, Xiaoyi Zhang, Richard Yuanzhe Pang, Clara Vania, Katharina Kann, Samuel R. Bowman

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Challenge:	Unsupervised pretraining has recently pushed the state of the art on many natural language understanding tasks.
Approach:	They perform a large-scale survey on a pretrained RoBERTa model with 110 intermediate-target task combinations and 25 probing tasks to reveal the specific skills that drive transfer.
Outcome:	The proposed model is trained on 110 intermediate-target task combinations and compared with 25 probing tasks to reveal the specific skills that drive transfer.

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples (2023.emnlp-main)

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

Copied to clipboard

Deqing Fu, Ameya Godbole, Robin Jia

Challenge:	Manually collecting challenging negative examples can be costly and domain-specific.
Approach:	They propose a method that synthesizes negative examples from only positive ones using a mask infilling model.
Outcome:	The proposed method closes 69.6% of the performance gap on a dataset where half of the evaluation examples are unanswerable compared to a model trained on SQuAD 2.0.

Are Visual-Linguistic Models Commonsense Knowledge Bases? (2022.coling-1)

Copied to clipboard

Hsiu-Yu Yang, Carina Silberer

Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach (2021.emnlp-main)

Challenge:	PTLMs are used to extract knowledge from text on demand.
Approach:	They compare visual-linguistic and language-only visual-language models in a zero-shot commonsense question answering inference task.
Outcome:	The proposed models are highly promising on certain types of commonsense knowledge associated with the visual world.

Copied to clipboard

Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Transfer Fine-Tuning: A BERT Case Study (D19-1)

Challenge:	Existing approaches to adversarial regularization treat adversarials and defending players equally, which is undesirable because only the defending player contributes to the generalization performance.
Approach:	They propose a method which formulates adversarial regularization as a Stackelberg game and induces a competition between a leader and a follower.
Outcome:	The proposed method outperforms existing adversarial regularization baselines on a set of machine translation and natural language understanding tasks.

Copied to clipboard

Yuki Arase, Jun’ichi Tsujii

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)

Challenge:	Recent advances in sentence representation learning have improved the performance of BERT models, but the computational power required is an obstacle preventing practical applications from adopting the technology.
Approach:	They propose to inject phrasal paraphrase relations into BERT to generate suitable representations for semantic equivalence assessment instead of increasing model size.
Outcome:	The proposed model improves a smaller model while maintaining the model size.

Copied to clipboard

Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen

Challenge:	a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages.
Approach:	They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models.
Outcome:	The proposed benchmarks show that the current models perform worse than the human ceiling.

ADEPT: An Adjective-Dependent Plausibility Task (2021.acl-long)

Copied to clipboard

Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, Jackie Chi Kit Cheung

The Unstoppable Rise of Computational Linguistics in Deep Learning (2020.acl-main)

Challenge:	ADEPT is a large-scale semantic plausibility task that requires a significant degree of world knowledge and common-sense reasoning.
Approach:	They propose a large-scale semantic plausibility task that pairs 16 thousand sentences with slightly modified versions obtained by adding an adjective to a noun.
Outcome:	The proposed task is easier for humans (85% accuracy), but more difficult for transformer-based models (71% accuracy).

Copied to clipboard

James Henderson

Challenge:	a quarter century ago, linguists assumed that language knowledge needed to be innate . but vector-space representations and machine learning algorithms are much more powerful than was thought .
Approach:	They trace the history of neural networks applied to natural language understanding tasks . they argue that Transformer is not a sequence model but an induced-structure model .
Outcome:	The proposed model is not a sequence model but an induced-structure model, the authors argue . they argue that the nature of language has had a profound impact on progress in machine learning .

Language Detoxification with Attribute-Discriminative Latent Space (2023.acl-long)

Copied to clipboard

Jin Myung Kwak, Minseon Kim, Sung Ju Hwang

Commonsense Knowledge with Negation: A Resource to Enhance Negation Understanding (2026.findings-acl)

Challenge:	Existing methods to detoxify toxic text require excessive memory, computations and time.
Approach:	They propose a method to generate toxic text using an attribute-discriminative latent space.
Outcome:	The proposed method outperforms baselines on detoxified language and dialogue generation tasks while being time- and memory-efficient.

Copied to clipboard

Zijie Wang, MohammadHossein Rezaei, Farzana Rashid, Eduardo Blanco

Is Language Modeling Enough? Evaluating Effective Embedding Combinations (2020.lrec-1)

Challenge:	Negation is a common and important semantic feature in natural language, yet Large Language Models struggle when negation is involved in natural learning tasks.
Approach:	They propose to augment existing corpora with negation by automatically augmenting existing ones with negations by combining multiple triples with if-then relations.
Outcome:	The proposed approach yields two new corpora containing over 2M triples with if-then relations.

Copied to clipboard

Rudolf Schneider, Tom Oberhauser, Paul Grundmann, Felix Alexander Gers, Alexander Loeser, Steffen Staab

Information Extraction & Text Mining

Challenge:	specialized embeddings are not available for tasks like entity linking or paragraph classification.
Approach:	They evaluate whether universal embeddings can be complemented by specialized embeddables.
Outcome:	The proposed embeddings outperform state-of-the-art embeddables without any fine-tuning.

Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistant (2025.findings-emnlp)

Copied to clipboard

Lei Shen, Xiaoyu Shen

Challenge:	Auto-SLURP is a benchmark dataset for evaluating multi-agent frameworks powered by large language models.
Approach:	Auto-SLURP is a benchmark dataset aimed at evaluating LLM-based multi-agent frameworks . authors propose it extends original SLURP dataset by relabeling data and integrating simulated servers and external services.
Outcome:	The proposed dataset extends the original SLURP dataset for natural language understanding tasks.

Fast Adaptation via Prompted Data: An Efficient Cross-Domain Fine-tuning Method for Large Language Models (2024.lrec-main)

Copied to clipboard

Yiming Zhang, Hantao Yang, Haobo Wang, Jake Zhao

EventOA: An Event Ontology Alignment Benchmark Based on FrameNet and Wikidata (2023.findings-acl)

Challenge:	Large language models (LLMs) have been successful in a variety of natural language understanding tasks, but domain discrepancies between the downstream task and the pre-training corpora may have hindered LLMs to excel further in the vertical applications.
Approach:	They propose a Fast Adaptation method for LLMs via Prompted Data that integrates downstream text corpora, gold labels and external knowledge sources into a highly controllable prompt.
Outcome:	The proposed method bridges the gap between the downstream task and the pre-training corpora and integrates downstream text corpors, gold labels and external knowledge sources into a highly controllable prompt.

Copied to clipboard

Shaoru Guo, Chenhao Wang, Yubo Chen, Kang Liu, Ru Li, Jun Zhao

Knowledge Representation

Challenge:	Existing studies on event ontologies focus on entity-based OA, and neglect event-based one . however, independent development of event ontoologies often results in heterogeneous representations that raise the need for establishing alignments between semantically related events.
Approach:	They propose a multi-view event ontology alignment method that utilizes description information and neighbor information to obtain richer representations of the event ontoologies.
Outcome:	The proposed method outperforms existing entity-based methods and can serve as a strong baseline for future research.

NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms (2024.acl-long)

Copied to clipboard

Jonathan Zheng, Alan Ritter, Wei Xu

Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024.acl-long)

Challenge:	Prior work on temporal language change observed degradation when finetuning on older text and evaluating on newer data and named entities.
Approach:	They construct a benchmark to evaluate LLMs’ ability to generalize to neologisms with various natural language understanding tasks and model perplexity.
Outcome:	The proposed model performs better in downstream tasks and with later knowledge cutoff dates than models with earlier knowledge cut off dates.

Copied to clipboard

Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, Michael Bendersky

Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022.emnlp-main)

Challenge:	Recent advances in natural language processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs).
Approach:	They propose an Explanation-Aware Soft Ensemble framework to empower in-context learning with Large language models.
Outcome:	The proposed framework can be used to enhance in-context learning on seven natural language understanding tasks and four varying-size LLMs.

Copied to clipboard

Yingxiu Zhao, Yinhe Zheng, Zhiliang Tian, Chang Gao, Jian Sun, Nevin L. Zhang

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation (2021.emnlp-main)

Challenge:	Existing generative replay methods use only a single task-specific token to control their models.
Approach:	They propose a method to capture task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation.
Outcome:	The proposed method outperforms baselines on natural language understanding tasks of advanced task-oriented dialogue (ToD) systems.

Copied to clipboard

Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Dan Garrette, Graham Neubig, Melvin Johnson

Self-training with Few-shot Rationalization (2021.emnlp-main)

Challenge:	Recent advances in multilingual natural language processing have improved performance on benchmarks such as XTREME and XGLUE by 13 points . however, improvements have been easier to achieve in some tasks than others .
Approach:	They extend XTREME to XTRAME-R, which includes ten natural language understanding tasks and covers 50 typologically diverse languages.
Outcome:	The proposed framework improves the performance on the XTREME multilingual benchmark by 13 points compared to human-level performance on English transfer learning.

Copied to clipboard

Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata Mukherjee

KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning (2024.lrec-main)

Challenge:	Recent work focused on training largescale and complex neural network models, but they are opaque in terms of their decision-making process.
Approach:	They propose a multi-task teacher-student framework for self-training pre-trained language models with limited task-specific labels and annotated rationales.
Outcome:	The proposed model improves performance in low-resource settings by making it aware of its rationalized predictions.

Copied to clipboard

Dongyang Li, Taolin Zhang, Longtao Huang, Chengyu Wang, Xiaofeng He, Hui Xue

Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models (2024.emnlp-main)

Challenge:	General pre-trained language models (PLMs) leverage relation triples from knowledge graphs (KGs) and integrate external data sources into language models via self-supervised learning.
Approach:	They propose to learn Knowledge-Enhanced language representations with Hierarchical Reinforcement Learning (KEHRL) to detect positions for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge.
Outcome:	The proposed model can detect essential positions in texts for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge.

Copied to clipboard

Nisarg Patel, Mohith Kulkarni, Mihir Parmar, Aashna Budhiraja, Mutsumi Nakamura, Neeraj Varshney, Chitta Baral

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	Existing logical reasoning evaluation benchmarks focus on simplistic single-step or multi-step reasoning with limited set of inference rules.
Approach:	They propose to use a multi-step logical reasoning evaluation dataset to measure their ability for human-like multi- step logical thinking.
Outcome:	The proposed dataset covers three logic types including propositional, first-order, and non-monotonic logic with various inference rules and depths.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Language Fusion for Parameter-Efficient Cross-lingual Transfer (2025.acl-long)

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Copied to clipboard

Philipp Borchert, Ivan Vulić, Marie-Francine Moens, Jochen De Weerdt

Question Answering

Challenge:	Limited availability of multilingual text corpora for pretraining results in poor performance on downstream tasks due to undertrained representation spaces for languages other than English.
Approach:	They propose a method that integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations to enhance representation quality and transfer performance for languages other than English.
Outcome:	The proposed method improves representation quality and performance for languages other than English while maintaining parameter efficiency.

Soft Head Selection for Injecting ICL-Derived Task Embeddings (2026.findings-acl)

Copied to clipboard

Jungwon Park, Jimyeong Kim, Changin Choi, Wonjong Rhee